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Abstract: A perturbative new family of fermions is now severely constrained, though 
not excluded yet. We reconsider the current bounds (i.e., direct and from Higgs searches, 
Rf,, oblique parameters) on the fourth generation parameter space assuming the case of a 
small CKM mixing with the third generation. We identify viable scenarios featuring either 
a light or a heavy Higgs boson. A set of representative benchmark points targeted for 
LHC searches is proposed with a normal (inverted) quark mass hierarchy where t' — )• b'W 
(6' — )• t'W) are sizable. In the case where the fourth generation couplings to the lighter 
quark families are small, we suggest that search strategies at the LHC should include both 
pair (strong) and single (weak) production with bb+nW (n = 2, 6) final state signatures. 
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1 Introduction 

It is generally claimed that the Standard Model (SM) comprises three generations of 
fermions. However, many fundamental problems do not find an answer into this framework, 
and the possibility of additional massive fermions, such as a new sequential family of quarks, 
is currently among the models in the spotlight of experimental searches at the LHC (for 
an overview, see, e.g., [1]). Besides its phenomenological relevance, the Standard Model 
with a fourth generation (hereafter SM4) can also serve as a template for new physics 
models for which the unitarity of the 3x3 CKM matrix might be violated [2]. Despite 
the absence of hints in experimental searches, there is currently a renewed interest in the 
fact that new fermion families are still allowed by the electroweak precision constraints if 
additional parameters are considered. For instance, considering non-zero mixings between 
a new heavy doublet and the lighter quark generations has recently been shown to widen 
the allowed parameter space [3]. Furthermore, the mass difference between the new heavy 
quarks also requires specific attention. In their seminal paper [4], Kribs et al. proposed the 
correlation 



between the SM4 Higgs boson mass and the quark mass difference, Anigi = mt'—mhi. While 
simple, this approximation is known to overconstrain the fourth generation parameter space 
[5], and the experimental searches now require more refined predictions. 
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Table 1: Summary of the relevant fourth generation parameters for the LHC searches, with 
the fourth generation fermions q' = t',b' and /' = r', i^' (Am;/ = m^' — itIt-'). The marks 
denote the dependencies on the constraints considered in our analysis. 

In this work we argue that a better description of the fourth generation parameter 
space is provided by 

A^g' < (1 + ^ In ^^^^^^ - 15 4) X rnw, (1-2) 

where S34 = sin034 = \Vt'b\ is the CKM mixing element between the two heaviest quark 
families. 

In the following, we scrutinise the various SM4 scenarios uncovered by the current 
approximations, consistently with the upper limit (1.2). In Section 2, we review the cur- 
rent bounds on the relevant fourth generation parameters in collider searches, taking into 
account the possibility for small but non-zero CKM mixings with the third family. Section 
3 describes our numerical analysis of the fourth generation parameter space, allowed by 
the electroweak precision observables. Considering the relative importance of the quark 
mass splittings, we present a selection of representative benchmark points for the normal 
{mf > mf,' ) and inverted {rrif < mi,' ) quark mass hierarchies with a particular focus on the 
light Higgs scenario. In Section 4, we detail the production and experimental signatures 
corresponding to these scenarios, and work out the associated cross-sections and decay 
rates. Our conclusions are presented in Section 5. 

2 Current bound dependencies on the fourth generation parameters 

While strong pair production at the LHC only depends on the new heavy quark masses, 
the electroweak production and decay modes are mainly sensitive to their mass difference 
and CKM mixings. As our purpose is to provide representative benchmark scenarios, the 
relations between the various SM4 parameters must be investigated carefully. Following 
Table 1, we first analyse the direct limits on the fourth generation fermion masses. The 
Higgs direct hints provide an independent constraint on the allowed parameter space due to 
the decoupling of the new heavy fermion masses. The electroweak precision parameters are 
then considered, with the non-oblique correction to the Z — )• 66 vertex setting a stringent 
upper bound on S34 as a function of the quark masses. Finally, the oblique parameters S 
and T are used to derive (1.2) as a function of all previous constraints. 
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2.1 Direct limits 

CDF and DO experiments previously set the strong limits mf > 335 GeV [6] and mj,' > 385 
GeV [7] at the 95% confidence level (CL) for the masses of new up- and down-type heavy 
quarks. Yet, direct searches at the LHC now hint at even more stringent constraints. 
Searching for short-lived b' quarks in the signature of trileptons and same-sign dileptons, 
CMS ruled out mf,/ < 611 GeV at 95% CL assuming BR(6' tW) ~ 100 % [8]. A search 
for heavy pair-produced top-like quarks has also been conducted considering t' — t- bW as a 
prompt exclusive decay. No excess over the SM expectations has been observed, excluding a 
t' quark with a mass below 557 GeV [9] . The inclusive search [10] also set a strong limit on a 
degenerate fourth generation, > 495 GeV at 95% CL, assuming a minimal off-diagonal 
mixing between the third and the fourth generation. Recently, the ATLAS collaboration 
ruled out mq < 350 GeV at 95% CL by searching for pair-produced heavy quarks QQ in 
the two-lepton channel [11], with BR{Q — )• qW) ~ 100 %, where q = u,d, c, s, b. This last 
analysis does not rely on b-tagging and provides a more conservative bound, which we will 
use in the following. 

On the other hand, we observe that most of the aforementioned limits are model- 
dependent, so that some regions of the parameter space are still allowed and the associated 
(peculiar) phenomenology unexplored. Firstly, direct searches rely on the hypothesis that 
new heavy quarks are close in mass and mix predominantly with the third generation, which 
implies short lifetimes and prompt decays in the beampipe. While the former condition is 
well motivated by the precision electroweak measurements, a small yet non-zero 3-4 family 
CKM mixing element must be considered very carefully. Secondly, the current limits 
now reach the bound nifi < 550 GeV required from the unitarity for the partial S-wave 
amplitude in t't' scattering at tree-level [12]. Additionally, perturbative methods could 
become inadequate. As shown in [13] from the 2-loop renormalisation group equations, 
the fourth generation Yukawa couplings lead to the breakdown of perturbativity below the 
TeV scale if mt'^b' > 375 GeV. In summary, with the forthcoming searches pointing at even 
larger mass scales, the possibility for long-lived heavy fermions should be considered, as 
well as a strongly coupled fourth generation or Yukawa bound states [14-17]. 

Regarding the fourth generation lepton sector, the LEP obtained the lower bound 
rriT-i > 100 GeV for new heavy charged leptons [18]. Assuming Dirac masses, the Z invisible 
width set nii,' > 45 GeV for new heavy stable neutrinos [19]. Although the off-diagonal 
lepton mixings are required to be smaller than 0.115 to be consistent with the recent 
global fits [20], a trivial 4x4 PMNS unitary matrix is considered throughout this analysis, 
namely a long-lived Dirac neutrino with |?7,y'r'| = 1- We mention, however, the interest of 
the Majorana the exhaustive study [21] has shown. 



-3- 



2.2 Higgs search constraints 

The recent results from the CMS experiment (with integrated luminosity of 4.8 fb~^) now 
exclude a SM Higgs boson mass between 127.5 GeV and 600 GeV at 95% CL with all 
channels combined [22]. A local significance of 3. la is obtained for a 124 GeV SM Higgs 
boson decaying to two photons [23]. ATLAS observed an excess of events around rriH = 125 
GeV (126.5 GeV) with a local significance of 2.5o- {2.8a) in H ZZ* Al {H 77), 
constraining mn in the ranges 117.5 — 118.5 GeV and 122.5 — 129 GeV with an integrated 
luminosity of 4.9 fb"^ [24-26]. As far as a fourth family of fermions is considered, CMS 
recently excluded a SM4 Higgs boson in the mass window 120-600 GeV at 95% CL [27], 
accounting for ~ 600 GeV heavy fermion masses and in agreement with the spectrum 
(1.1). Under similar assumptions, ATLAS rules out a fourth generation SM Higgs boson 
with mass between 119 and 593 GeV at the two sigma level [28]. 

While very little room seems to be left for a light SM4 Higgs boson, recent studies 
emphasised that the above exclusion ranges did not take into account the possibility for 
a stable fourth generation neutrino [29, 30]. If such new neutral leptons are lighter than 
half the Higgs boson mass, the opening of the new invisible mode H ^ u'U' increases the 
Higgs total width, and overtakes the H — t- WW^*\ Z Z^*^ and // rates with a substantial 
branching ratio in the low mass region. However, it is straightforward to check that the 
recent hints of a 125 GeV scalar boson fine tune m^i to a value very close to half the Higgs 
boson mass. Given that H ^ bb is the dominant visible signal for a light SM-like Higgs 
boson with respect to H ^ v'v' in SM4, the ratio 



must be of order unity. Combining (2.1) with the assumption that the Higgs invisible 
branching ratio does not excess 50%, the new neutral lepton mass must lie between 61 and 
62.5 GeV. Were direct searches to exclude rriyi < rriH/'i, the Higgs decay patterns would 
not be affected by the neutrino mass, and the aforementioned exclusion ranges apply. If 
rriyi > mH/2, a light SM4 Higgs boson is expected to lead to significantly more events. In 
particular, the H — t- // and H — t- ZZ^*^ signal strengths are enhanced by a factor 5 to 6 
for niH — 125 GeV, given the gluon fusion enhancement [31]. This remains consistent with 
the recent LHC results, except in the channel if — t- rr, for which the excess observed by 
CMS is currently inconsistent with the SM4 predictions [27], but is still to be confirmed 
by the ATLAS results. 

A new family of fermions implies important consequences for collider phenomenology, 
as it significantly modifies the Higgs production and decay rates with respect to SM. At 
the leading order, the Higgs production cross-section via gluon-gluon fusion increases by a 
factor of about 9 due to the additional t' and b' fermion loops arising from the decoupling 
of the fourth generation quark masses [4] . The same enhancement in H ^ gg increases the 
Higgs total width by a factor of about 1.6. On the other hand, the if — )• 77 width suffers 
a factor of about 5 suppression due to the new heavy fermion loops involving t', b' and r'. 
This accidental cancellation against the W contribution in the -ff — )• 77 amplitude results 
in a comparable number of diphoton events. 




(2.1) 
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Recently, the calculation of the NLO electroweak corrections hinted at very large ra- 
diative corrections to the Higgs decay amplitudes in SM4. Considering fourth generation 
fermion masses larger than 450 GeV (600 GeV), the Higgs branching ratios into WW and 
ZZ pairs are suppressed by corrections of the order of -40% (-60%), whereas BR(fZ' — )• //) is 
enhanced by 30% (60%) in SM4 [32]. Additionally, the rate a{gg ^ H)x BR{H jj)\sm4 
is further suppressed by one order of magnitude with respect to the LO result, which hardly 
accomodates the fourth generation fermion scenario [33-35]. However, such an accidental 
difference between the LO and NLO corrections raises legitimate concerns about the stabil- 
ity of the above predictions. Therefore, this, together with the fact that the 77 signal might 
still be a statistical fluctuation, suggest us to leave open the light Higgs window between 
117 GeV and 135 GeV, i.e., up to where the decay mode into a bb pair is dominant. 

Finally, we emphasise that if the hints for a SM-like light scalar signal would be due 
to statistical fluctuations and disappear with more integrated luminosity at the LHC, the 
possibility for a heavy Higgs would need to be reconsidered. Although the CMS and 
ATLAS experiments rule out mn < 600 GeV in the context of SM4, it has been noted 
in [36] that the use of the narrow width approximation to set exclusion limits on a SM 
Higgs boson should be avoided for larger mass values. In particular, the effects due to 
off-shell production and decay can lead to large deviations with respect to the zero-width 
approximation for Higgs boson masses larger than 300 GeV. Given that a proper description 
of the Higgs line shape is required to set more accurate exclusion limits at such scales, the 
conservative lower bound uih > 450 GeV will be considered in the following for the high 
mass region. 

2.3 Electroweak precision observables 

The electroweak precision observables are well known to yield stringent limits on the possi- 
ble extensions of the SM, and set up compelling bounds on the allowed mixings between a 
sequential fourth generation and the lighter fermion families. Considering a generic flavour 
structure, the addition of a new quark doublet requires to extend the CKM mixing sector 
to a 4 X 4 unitary matrix, whose entries depend upon three additional mixing angles and on 
two new CP- violating phases. Assuming a non-zero 3-4 family CKM mixing, the Rh ratio 
T(Z — )• bb) /T{Z — )• hadrons) can be shown to provide a solid constraint on a non-degenerate 
fourth generation, independently of the Higgs and leptonic sectors. On the one hand, the 
non-oblique corrections to Rb are known to remain below several permille for m^/ < 100 
GeV [37] . On the other hand, the nondecoupling effects present in the Zbb vertex turn out 
to be slightly sensitive to non-zero mf — rrn,/ quark mass splittings due to non-negligible 
0{a1) QCD corrections. Following the derivation given in [38] and [39], their effect on 
the Z decay rate depends on the two quantities 5h and ^Iqc£)- While vanishing for light 
quarks, is sensitive to t and t' loop corrections in Z — t- 66, and reads 




(2.2) 
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Conversely, S^q^jj^ includes the QCD contributions to the axial part of the decay. As 
computed in [40], the induced axial and vector currents corrections to the Z boson decay 
rate receive significant O^a^) corrections due to the possible large splitting of heavy quark 
doublets. They stem from the absorptive part of the diagram obtained by squaring the 
tree-level decay Z ^ qq with the real emission of 2 gluons Z ^ qq ^ gg via a triangle 
loop. In the context of four quark generations, the incomplete cancellation between the 
various heavy doublets gives rise to 



f Q) f f OLg \ ^ 

Hqcd = - -2,-2 (~) fif^t) + at' fint') - fil^b')] (2.3) 



vj + aj\7r 

where Os is the QCD coupling constant evaluated at the Z pole (i.e. as{M'^) = 0.119) 
and aq = 2/|, Vq = 2/| — AQqS^^ are the vector and axial vector couplings of the q quark, 
respectively. The quark masses rfij are defined in the MS scheme, with = 4m?/M^. 

The function / reads [39] 

. , / 4 \ 0.346 0.211 , , 

The S:[qqj^ correction depends explicitely on both fourth generation quark masses, providing 
an additional constraint on the 3-4 family mixing as a function of the mass splitting. Using 
the exact expressions given in [38], we obtain 

_ 1.594(1 + (^,) 
^'" 5.722 + 1.594(1 + 5,) ^ 

for rrit' = 500 GeV, which we can compare to the experimental value R'^'^ = 0.21629 ± 
0.00066 [19]. We estimate that 6{q^^ yields a -5% (+4%) correction for mf = 500 GeV if 
ArUqi = 80 GeV (—80 GeV), which effect becomes weaker for larger mf masses. While this 
implies a measurable effect in Rb, it depends sensitively on our choice of the MS top quark 
mass, here taken as fht = 163.5 GeV. Combining (2.5) with the condition Rb > i?^^^ at 
the 2a level, we obtain the following upper bound on the 3-4 CKM mixing (for Arriq' = 
GeV) : 

' 0.17 if mt' = 400 GeV 
0.13 if mt/ = 500 GeV (2.6) 
0.11 if mt' = 600 GeV 



which can be approximated by 



Mr, < 0.8 ^. (2.7) 

rrif 

Additionally, such bounds occur to be sensitive to the quark mass splitting when accounting 
for the 0{a^) QCD corrections in the Z ^ bb decay rate. Most important, larger quark 
masses favour smaller t — b' and t' — b mixings, independently of the Higgs boson and heavy 
neutrino masses. 
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Interestingly, the oblique parameters originally introduced by Peskin and Takeuchi 
[41] lead to similar, albeit less model independent conclusions. Including all one-loop self- 
energy contributions of the electroweak vector bosons W, Z and 7 arising from new heavy 
chiral fermions, the S, T and U have been evaluated in the presence of a perturbative 
fourth family of fermions in [42]. Assuming that the corresponding new physics scale is 
much larger than the electroweak scale and that additional couplings to the SM fermions 
are suppressed, the contributions from a fourth fermion family to the oblique parameters 
read 

, mi,i m,,i \ 

2 + In-? (2.8) 



Stt V mt' rrij 

rp _ 3 [ml + ml + sl^{Ft>b' - Fg') - Ft^b'] + [ml, + m^, - F^/^/] 

where sw and cw define the sine and cosine of the weak mixing angle respectively, and 

2m,^m? 

Fij= / (2.10) 
mf — m, m- 

We observe that large quark mass differences increase the T4 contribution, while small 
S34 values balance this effect. Assuming |m^/ — m,y/| < 90 GeV and mf ^ 500 GeV, 
these observations can be summarised by the approximate upper bound (1.2) which the 
T parameter constraint translates to if T < 0.3 is fulfilled. As depicted in Figure 1, it 
extends the approximation (1.1) and includes the mixing angle dependence between the 
two heavier generations, setting an upper limit on the allowed quark mass splitting from 
the T parameter. 



\mt-mti\ (GeV) 




/"H (GeV) 



Figure 1. 95% CL upper bound on |Amq/| versus mn for mt' = 500 GeV as given in (1.2) from 
the T parameter constraint, for the mixing values S34 = 0.1 (black), 0.05 (red) and 0.01 (blue). 
The dashed line depicts the value of the quark mass splitting as defined by the relation (1.1). No 
significant change is observed below S34 < 10^^. 
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The corrections to the U4 parameter are known to be neghgible if T4 < 0.4 [42], so that 
we restrict our analysis to the two dimensional S — T parameter space in the following. The 
54 parameter is sensitive to the mass scale of the new chiral fermions, whereas T4 constrains 
the mass splitting between all isospin partners. Accordingly T4 increases with S34, while 5*4 
is CKM-independent. Besides the fermion contributions, a heavy Higgs boson also affects 
the oblique corrections, which are known to depend logarithmically on the Higgs mass. 
In our analysis, the complete corrections computed at the Z pole given in [43] have been 
used. If larger than the SM reference, the Higgs boson mass niH simultaneously lowers 
down the T parameter and increases S. On the other hand, a smaller 3-4 mixing reduces 
the T parameter for a heavy Higgs mass without prejudicing the S parameter. 

While small values are favored in the light Higgs case, fermion mass splittings larger 
than the W mass remain allowed in significant regions of the SM4 parameter space if 
niH > 450 GeV. In the presence of a finite 3-4 CKM mixing element, the nondecoupling 
corrections to S and T imply a strong correlation between the fourth generation mass scale 
and the largest allowed value for \ Vt'b\ — S34. The contour regions in the S — T plane for a 
given confidence level CL are given by 

2/c (5'- 5*0)^ (T-To)^ 2{SoT + STq - ST - SqTq) /oti^ 
X {S,T) = ^- ^ + ^7:^ ^ + 71 2\ < -21n(l-CL . 2.11 

As an illustration, we compute the corresponding confidence levels for S and T, by 
naively varying the quark masses {rati, my) within the range [300, 800] GeV. Only evaluated 
for the Higgs reference mass rri^^^ = mz-, the LEPEWWG fit results Sq ^ as = 0.05 it 
0.10 and Tq ± ar = 0.10 ± 0.09 are used for the central values and their related standard 
deviations (p = 0.85 gives the correlation coefficient between the two parameters) [44]. 
The top reference mass is defined as ml'^^ = 173.2 GeV. All relevant quantities, including 
a"^(Mz) = 128.89 and s^(Mz) = 0.22291, are considered at the Z pole. We have checked 
that our results are in agreement with [42] at the permille level, yet very sensitive to the 
choice of {So,Tq). As a result, the upper limit on the sine of the mixing angle 6*34 yields 

{ss,)s,T < (2.12) 

at the 95% CL, in agreement with [45], though weaker than (2.7). The minimization of 
(2.11) with respect to the t' mass and the mixing angle is shown in Figure 2, as well as the 
upper bounds derived from Rf,. 

Although they provide additional insight on the SM4 parameters, we stress that flavour 
physics constraints are not considered in this analysis. As we neglect the 1-4 and 2-4 CKM 
mixing, the electroweak precision observables are found to give more severe constraints on 
the maximally allowed values for \Vt'b\- However, both the electroweak precision and the 
flavour constraints are known to favour small mixing with a heavy fourth family [42] . 
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Figure 2. 68% (darker blue), 95% (blue) and 99% (grey) CL allowed regions from (2.11) for the 
3 — 4 CKM mixing element as a function of mf, with all other SM4 parameters unconstrained. 
We find that the mixing between the third and fourth generation quarks is restricted such that 
S34 < 0.15 (0.075) holds at the 2a (Icr) level if rrit' > 500 GeV when considering the T parameter. 
The non-oblique correction arising from the Z ^ bb rate at the 95% CL is given by the solid black 
line for Am^/ = GeV. The upper and lower dashed lines give the corresponding constraints for 
Aruq' = —80 GeV and Aniq' = 80 GeV, respectively. 

3 Parameter space 

In this section we present a systematic analysis of the SM4 parameter space, assessing the 
possibility that particular regions might have evaded the experimental searches. Parameter 
sets minimizing the values of the electroweak parameters S and T are purposely identified, 
considering the relative importance of the possible fermion mass hierarchies for the case 
of vanishing 2-4 and 1-4 family quark mixings. In this case, the mixing angle between the 
third and the fourth generation of quarks fully encodes the new flavour sector, and provides 
a simple baseline scenario to constrain efficiently the fourth generation scenarios probed at 
the LHC. We require in particular S34 < 0.13, given that such an upper bound is expected 
if the fourth generation quark masses are larger than 500 GeV. For much smaller mixings, 
the lightest fourth generation quark and lepton might be quasi-stable. Such long-lived 
fermions would then have distinct phenomenological signatures, and the current bounds 
should be re-examined [46]. 
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As far as the scalar sector is concerned, the experimental bounds on the SM4 Higgs 
boson mass discussed in the previous section severely restrict the allowed parameter space. 
We select purposely two regions of interest for the Higgs mass, namely 

(a) 117 GeV < mn < 135 GeV, 

(b) 450 GeV < nin < 700 GeV. 

We show in Figure 3 the scatter plots of the S and T parameters, scanning the SM4 
parameter space for these two scenarios. Assuming a normal quark mass hierarchy (m^/ > 
mfo/), we consider mt>^b' e [350,700] GeV, m^/ G [60,120] GeV and m^/ G [100,250] GeV, 
and a light and a heavy Higgs separately. 
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Figure 3. 5 and T ellipses corresponding to Aniqi < 50 GeV (left) and Anig/ > 50 GeV (right) 
for the 68%, 95% and the 99% contour regions as defined by the inequality (2.11). Each scatter 
plot depicts 10^ random points for scenarios (a) in red and (b) in blue, under the constraints 350 
GeV< mt'M < 700 GeV, 60 GeV< < 120 GeV, 100 GeV< m^' < 250 GeV and < S34 < 0.13. 



Favored by the electroweak precision fits, the case (a) is of first importance for the 
on-going searches at the LHC if the present excess in — >• 77 is a statistical fluctuation. 
On the other hand, (b) covers the allowed high Higgs mass region, whose upper limit 700 
GeV is chosen in agreement with the quantum triviality bound obtained for four chiral 
families, recently updated in [12]. As our results indicate, none of the two quark mass 
differences Anigi < 50 GeV and Aniqi > 50 GeV is disfavored if a small mixing S34 is 
required. Although the light Higgs case (a) is preferred at the 68% confidence level, the 
scenario (b) is still consistent at 95% CL with a fourth generation of fermions. Both sets 
of points overlap with a significant area of the confidence level ellipses, still allowing for a 
large parameter space. The case mf < nib', not shown on the above picture, shifts the sets 
upwards by AT ~ 0.035. 
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Regarding the Higgs mass dependence, the heavy case uih — 650 GeV is shifted 
towards AS ~ 0.08 and AT ~ —0.24 with respect to the hght scenario mn — 125 GeV. 
Regarding the possible mass hierarchies, the correlations between the fermion splittings 
ArUg/ = nit' ~ w-fe' ^-iid Ami' = i^u' ~ "^r' have been considered in [5] for various quark and 
lepton mass scales. For completeness, we briefly re-examine them here for a non-vanishing 
3-4 CKM mixing. Fixing rrifj = [125, 600] GeV, we plot in Figure 4 the corresponding 68%, 
95% and 99% CL regions in the {Aniq', Ami') plane for all possible mass differences and 
CKM mixing elements. 





1S0 



100 



> 

O 50 

^ 

-50 
-100 



-100 -50 50 100 

nip - lUf (GeV) 



-100 -50 50 100 



















■ \ 

















-100 -50 50 100 

ijij. - nil, (GeV) 



Figure 4. Contour regions in the Am^' — Am;/ plane as allowed from (2.11), for rriH — 125 GeV 
(top), 600 GeV (bottom), and S34 = 0.01 (left), 0.1 (right). The three confidence levels 68%, 95% 
and 99% are given by the purple, dark blue and light blue areas, respectively, for mt' = 500 GeV 
and ttIt' — 100 GeV. The vertical (horizontal) red lines define the ArUqi = mw (Am;/ = my/) 
threshold for the production of a real W int' ^ b' (r' -;-> u') decays. 
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Assuming a 500 (100) GeV scale for the fourth generation quarks (leptons), the x'^ 
values given by (2.11) allow large areas for Am^/, Am;/ > 50 GeV and S34 < 10~^. Numer- 
ically, a strong correlation is confirmed between the quark and lepton mass splittings, with 
a weak yet relevant dependence on the 3-4 CKM mixing. Given the logarithmic dependence 
of 5*4 on the ratio myij 7x1^-1, large lepton mass splittings can be allowed at the cost of a 
smaller quark mass difference balancing the T4 parameter. 

We observe that the lepton mass hierarchy rriyi > m,-' occurs to be strongly disfavored 
in view of these results, whereas the quark splittings Am^' > and Arugi < are equally 
preferred if m,yi < m^'. Assuming that S34 > 10"'^, small lepton mass splittings are allowed 
in scenario (a) if |Amg/| is larger than 50 GeV, while |Amg'| < 40 GeV defines a wide 
parameter region for which rrir' — m^i > 80 GeV is possible. The oblique parameters thus 
allow the two-body decay mode r' — )■ i^'W for fourth generation leptons if the quark mass 
difference is small. However, the constraints obtained from Higgs searches severely restrict 
such a possibility in the low mass region. 

Considering in particular the low mass region (a), we extract in Table 2 various repre- 
sentative benchmark points for rriH = 125 GeV and the two cases rrit' > my and m^/ < my. 
The heavy Higgs boson scenario (b) yields similar conclusions, though quark mass differ- 
ences larger than my/ are allowed at the 95% confidence level inasmuch as the conditions 
S34 < 0.05 and mu > 400 GeV are fulfilled simultaneously. Otherwise, quark mass split- 
tings larger than 75 GeV are disfavored if mn < 400 GeV, which confirms the non trivial 
correlation between the Higgs mass and the quark mass difference depicted in Figure 1. 

Little change is noticed with respect to these conclusions when considering mixing 
angles smaller than 10~^. As Figure 1 indicates, there is not much gain in lowering down 
S34 for the considered mass scales. We also observe that the upper bound on the quark 
mass splitting does not depend on any parameter other than the Higgs boson mass, unless 
the mixing angle saturates the upper bound required from i?^. If only the T constraint is 
satisfied, both the quark and the lepton mass scales can be increased without any effect on 
the S parameter. 
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Table 2 : Selected SM4 scenarios, allowed at the 95% CL by the S and T parameters, 
for mn = 125 GeV, and S34 = 0.1, 0.05, 0.01. The shown benchmark points give the largest 
possible quark mass splittings Am^^^ = mf—my, considering both the normal (mf > my) 
and the inverted (mt' < my) mass hierarchies. All masses are given in GeV, with m^i = 60 
GeV and Am^/ = 40 GeV. 
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4 Phenomenology at the LHC 



The predominant production and decay modes for a fourth generation of quarks strongly 
depend on the quark mass sphtting and hierarchy. Assuming a small 3-4 CKM mixing, we 
here consider the corresponding production modes, decay channels, and LHC signatures. 

4.1 Production 

Although pair production via the strong interaction is the main mechanism for producing 
new heavy quarks at the LHC, the electroweak production of a single top quark in the 
t-channel provides another relevant process to search for a fourth quark family [47]. For 
instance, the final state topologies expected from the electroweak t'b (t'b) and tb' {tb') 
production have been used recently in [10] to set very restrictive bounds on a degener- 
ate fourth generation. However, these processes scale by a factor |Vf/fep ~ \Vtb'\'^ and are 
strongly suppressed if the 3-4 CKM mixing element is negligibly small (while pair produc- 
tion is independent of it). Considering small S34 values as discussed in the previous section, 
single t' and b' production suffer strong CKM suppression, with cross sections below the 
femtobarn level for the considered mass scales and mixings. Yet, the t'b' (t'b') electroweak 
electroweak production in the t-channel provides an alternative without any CKM sup- 
pression, which overtakes t'b {tb') single production by two orders of magnitude if S34 is 
smaller than 10~^. 




(b) (c) 



Figure 5. Representative diagrams for the production of fourth generation quarks, (a) Strong 
pair production of heavy quarks at leading order ; (b) 2 — > 3 Born diagrams contributing to t'b' 
and t'b' electroweak production in the i-channel ; (c) t'b and tb' electroweak production (and the 
charge-conjugate processes) in the t-channel (suppressed if a small 3-4 CKM mixing is considered). 

Note further that two heavy particles are present in the final state, the interaction 
with the longitudinal W boson is proportional to the t' and b' Yukawa couplings and 
dominates the electroweak production of heavy quarks. This channel features the second 
dominant signal after strong pair production when the fourth generation mixings with the 
lighter quarks are negligible. Benchmark cross-sections at NLO have been tabulated in [47] 
for various fourth generation masses, PDF sets and LHC energies under the assumption 
|Vt'f,'P = 1. Tables 3, 4 and 5 present our predictions for relevant parameter choices. The 
t'b' (t'b') t-channel electroweak production, and the pair production cross-sections have 
been computed at 7, 8 and 14 TeV, with the CTEQ6.6 PDF set. The 6.0 version of the 
Monte-Carlo software MCFM [48] has been used for the calculations at NLO accuracy, 
following the conventions used previously in [47]. 
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Table 3: ^/s = 7, 8 and 14 TeV NLO predictions for the t'b' t-channel electroweak 
production, assuming |Vj'{i'p = 1. LO predictions are given in parentheses. The first 
uncertainty is calculated from the renormalisation and factorisation scales variation using 
the conventions detailed in [47] , while the second gives the upper and lower PDF errors for 
the CTEQ6.6 PDF set. The central value is computed for /i = /ij = = {fnt' + "ifc/)/2. 
The assumed CP invariance and chirality of the Wt'b' vertex lead to equal cross-sections 
when switching the t' and the b' masses, here given in GeV. 
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Table 4: Same as Table 3 for i-channel t'b' electroweak production, with the CTEQ6.6 
PDF set and \Vt>b'\'^ = 1- 



-14- 





7 TeV/ 


S TpV/ 

^qS (^^) 


^QQ (f^) 


400 


(955) 13291?^° l??o 


(1552) 21641341 +400 


(9206) 13033l^o^3_-ii69 


425 


(654) 907l^^^_t^f 


(1078) 14981?^^ t foi 


(6675) 94141^?^ l^°f 


450 


(454) Q29t\f^^\ 


(761) 10541J88+218 


(4913) 69051^61 +8J2 


475 


(320) 442+^5 +109 


(544) 75ll^^^_tjr 


(3665) 513lli? 1^^^ 


500 


(228) 314l^?i 


(393) 542l^°^_+^2^ 


(2767) 38611^^2 +5io 


525 


(164) 226+^f 


(287) 3951^2 +94 


(2112) 29381^^? 1^°« 


550 


(119) 1641^1 


(212) 29ll|3+73 


(1628) 2258+276 +329 


575 


(87.2) 120l?il?^ 


(158) 2161^^ li 


(1266) 1750l?i 


600 


(64.4) 88.4+?t 


(119) 162l?«l^^ 


(993) 1369li?^ 



Table 5: (LO) NLO cross-sections for Q = t',b' pair production at the LHC 7, 8 and 
14 TeV. The first error gives the factorization and renormahzation scales dependence, 
computed by setting fi = fij = fir and varying them between raqjl and Iraq for the upper 
and lower deviations from the central value \x = mg, respectively. The second error yields 
the CTEQ6.6 PDF uncertainties. 



4.2 Decay 

Large quark mass differences open new possibilities in the aforementioned benchmark sce- 
narios. Even if Imf — 'my\ < niw, the three-body t' — )• b'W^*'^ {b' —J- t'W^*^) decay can 
overcome the two-body qW modes at tree-level if the magnitude of the CKM entry Vfq 
{Vqb') at which the t' — )■ Wq {b' — )• Wq) decay proceeds is small enough. If the fourth 
generation quarks are allowed to decay into each other, the heavy-to-heavy transition 
t' b'W {b' t'W) competes with the CKM suppressed decay t' —f bW {b' — )- tW) for 
114/^1 ~ 1 14;,/ 1 ^ \ Vtih>\ [49]. Assuming that all off-shell particles are stable with a negligible 
width, the two-body decay width of a heavy quark t' is given by 

r(*'^ ,.,-•,) = ^m,//.(^.^), (4.1) 

/2(a,/3) = [(1 - + P\l + a^) - 2/3^] ^[1 - (a + /3)2][1 - (a - /3)2]. 

If rriti — my < mw (which defines the real W production threshold), t' — )• b'W^*'> {b' — )• 
t'W^*^) is phase-space suppressed, the partial rate (4.1) is not justified. Following [49], we 
require the t' (b') quarks to be on-shell particles in the final state for the heavy-to-heavy 
transitions. For a normal quark mass hierarchy {nit' > n^b'), the formula accounting for 
virtual and real W emission (F^y ^ with b' assumed on-shell) reads [46] 

/(a,/3,7) = 2 /"^'"^^'^^ [(i^)' + ^(l + /3)-2x']yA(i;^ 



[(1 - a2;)2 + 72] 

where A(a, b, c) = 0? + b"^ + — 2ab — 25c — 2ac. The case for nib' > fnf is similar. 
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We depict in Figure 6 the relative fraction r^/_j^^/^y(,) /rf/_5.{,vF versus the mass difference 
Arrigr for a normal quark mass ordering. The width effect occuring in the decay of a virtual 
W boson resonance near threshold is clearly visible for rrit' ~ myy + rrib' . Table 6 displays 
the decay rates and relative ratios obtained from (4.1) and (4.2), considering S34 = 0.1 
and 0.05. Under these assumptions, the decays t' — t- b'W and b' — t- t'W can compete 
with the 4 — 3 transitions for quark mass differences larger than m\Y, while the heavy-to- 
heavy transitions systematically dominate if S34 < 10~^ and Am^/ > 40 GeV. The relative 
fraction TfJ,^^,^r{*) /Ti,i^tw versus nib' — f^t' leads to similar results when considering the 
same mass scale. 




50 60 70 80 90 100 

m, -mf (GeV) 



Figure 6. Ratio of the t' h'W^*'^ and t' bW decay widths for a classical quark mass ordering. 
The case mf = 500 GeV is displayed, with \Vt'b\ = 0.2 (black), 0.1 (red), 0.05 (green) and 0.01 
(blue). 
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Table 6 : — )■ bW and t' — t- b'W^*^ decay widths in GeV, and their relative ratio for 
mf > nib' for various splitting scenarios and S34 = 0.1 (0.05). 
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4.3 Signatures at the LHC 

We now briefly discuss the possible decay signatures depending on their mass ordering 
for small S34 mixings. In the case of a normal quark mass hierarchy {mf > my), up- 
like fourth generation quarks either decay via t' — t- hW or cascade through t' — >• h'W — >• 
tWW —7- bWWW, with a real or a virtual W emission. The respective CKM mixing matrix 
factors \Vt'b\'^ and Vi;,'p|Vtbp being of the same magnitude, the t' — b'W* decay can 

dominate over the t' — )■ Wb mode, though r^,^^/^y(,) is strongly phase space suppressed 
[49]. If we enforce that BR(f b'W) > BR{t' bW) and postulate that t' and b' decay 
promptly within the detector, the most likely final states for a normal mass hierarchy are 

• (a) ft' b'b'W+W- tt 2W+2W- bb3W+3W-, 

• (b) b'b' tt W+W- bb2W+2W-, 

• (c) t'b' b'tW+W- tb2W+2W- bb?,W+2W- , 

• (d) t'b' b'tW-W+ ib2W~2W+ bb3W-2W+. 

Table 7 includes the branching fractions corresponding to these various signatures. 
Each of the above final states includes two 6-jets for singly and pair produced fourth 
generation quarks, while four to six W bosons occur in the decay of each signal event. These 
topologies all allow for same-sign (l^l^) and opposite-sign (l^P) dileptons in the final state, 
with substantial missing energy along the jets. Interestingly, t' pair production (a) provides 
a signal with up to 12 jets if all W bosons decay hadronically. Looking for the rare signature 
provided by two same-sign leptons with large missing momentum is then a relevant search 
strategy, though a small number of l^l^ events is expected. Except for b' pair production 
(b), we observe that the t' can be reconstructed from same-sign dileptonic events, as at 
least two same-charge W's follow from the decay chain t' — )• b'W — )■ tWW — )• bWWW. 

We also note that none of the above channels include same-charge W bosons decaying 
from the same b'. The corresponding final states include four W bosons and two b quarks. 
For mfi > my, the b' quark is not allowed to decay to t'W and the CKM suppressed 
b' — )• tW becomes the only possible decay mode. However, the requirement of two like- 
sign dilepton pairs provides this channel with a rare SM signature containing very few 
background events. Searching for a single lepton and trileptons is also a possible strategy. 

Although the relative cross section is expected to be smaller than pair production, t'b' 
electroweak production (c,d) features an interesting situation with up to 10 jets + 2 b- 
tags, for which an odd number of same-sign leptons can be expected, two of them decaying 
from the same t' quark. Although appropriate cuts can improve the discrimination, the 
overwhelming combinatorial background makes the reconstruction and selection challenging 
issues. 



-17- 



An inverted mass hierarchy (m^/ < mt') would feature a down- type quark decaying 
either as 6' — > t'W — > bWW or b' — )■ iVF — )■ bWW, leading to identical final states. The 
two corresponding decay rates are suppressed by | Vi'fe'p|T4';,p and |14f,'P| respectively, 
both proportional to S34. Assuming BR(6' — ^ t'W) » BR(6' — ^ tW), two to four W bosons 
can be present in the possible final states 

• (e) t'F bbW+W- 

• (f) b'V t'i' W-W+ bb2W-2W+ 

• (g) t'b' bi'W+W- bbW+2W- 

• (h) i'b' bt'W-W+ bb2W+W- 

We observe that both the t'i' and the b'b' pair production signals (e,f) are covered 
by the direct searches [8-11] in multiple channels under the condition BR(t' — >■ bW) = 
BR(6' — )• tW) = 100%. If a small s'^^ value is assumed, the above channels final states are 
however overwhelmed by the SM background, weakening the current bounds on and 
rrif,'. The t'b' production processes (g,h) allow for 3 W bosons and 2 b-tags in the final 
state. Despite the fact that the associated cross-sections are low and hardly competitive 
with pair production, the high-jet multiplicity provides again an interesting signature with 
the possibility of single leptons, same-sign dileptons, and trileptons in the final state. 
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Table 7 : Branching Ratios (in %) as a function of the number of leptons (/ = e, /i) 
for the various final states, under the assumptions BR(t' — )• b'W) ^ BR{t' — )• bW) and 
BR(6' ^ t'W) > BR(6' ^ tW). 
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5 Summary 



In this work, we reconsidered the available parameter space for a perturbative fourth gener- 
ation of fermions, as allowed by the current bounds, namely direct and from Higgs searches, 
as well as by the electroweak precision observables. Our results show that the relation (1.1) 
overconstrains the allowed SM4 scenarios if a small but non-zero CKM mixing between the 
two heaviest quark families is required. Considering the bound (1.2) and accounting for 
large mass splittings, we find that the heavy-to-heavy transitions t' — t- Wh' {h' — )• t'W) 
provide relevant decay modes if the 3-4 family CKM mixing element is smaller than 0.1. 
Additionally, we confirm that the Z — t- 66 decay rate gives a stronger bound on the 3-4 
family CKM mixing element than the oblique parameters, independently of the Higgs and 
lepton sectors, and such that the t' — )• hW and 6' — )• tW decays are significantly suppressed. 

The tantalizing hints of a light Higgs boson at the LHC in the 77 channel, if confirmed, 
would exclude a fourth generation of fermions. However, the large suppression of the ^^77 
coupling due to the novel large Yukawa couplings casts some doubts on the perturbative 
validity of the NLO electroweak corrections. With more statistics, the comparison with 
the SM expected rates in the modes H — > WW^*\ H — > ZZ^*^ and H ^ ff should 
allow to conclude whether the confirmed evidence of a light Higgs boson is incompatible 
with a fourth generation. While the Higgs mass range 120-600 GeV is claimed to be 
excluded below the two sigma level from the recent CMS and ATLAS results, we argue 
that niH > 450 GeV requires more scrutiny given the large width of the Higgs boson in 
this range. Given the importance of excluding a larger spectrum from the direct searches, 
we have presented scenarios which could have evaded the experimental analyses. Even if 
t't' {h'V) pair production is unaffected by the magnitude of the fourth generation mixings, 
the possible importance of the heavy-to-heavy transitions t' — )• 6' and 6' — )• t' opens new 
challenging channels for the forthcoming searches if a very small 3-4 family CKM mixing 
element is allowed. For extremely small values of this 3-4 mixing element, long-lived quarks 
and possibly bound states evade the direct searches, and require dedicated analyses [46]. 

In particular, the t'V and i'b' electroweak productions in the t-channel can overcome 
t'h and th' single production, and be accessed in the forthcoming searches. Allowing for 
signals with a higher jet multiplicity and same-sign dileptons, the subsequent decays can 
provide a clean signature against the SM background in all corresponding final states. These 
scenarios remain crucial to settle the question whether a perturbative and sequential fourth 
generation can be ruled out from direct searches at the LHC. In the case where the fourth 
generation couplings to the lighter families are small, we suggest that search strategies 
at the LHC should include both strong and weak production with multiple W final state 
signatures. 
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Note added: While this work was being completed, a related analysis by Dighe et al. 
has appeared in [50], in which values of mn as large as 800 GeV are considered. While 
similar results are obtained in this particular case, we also give a more specific focus on 
the light Higgs scenario, taking into account the current bounds for ran = 125 GeV. In 
this particular case, representative benchmark scenarios are provided from a systematic 
scan of the SM4 parameter space, consistently with the electroweak precision constraints. 
Assuming a small 3-4 CKM mixing, we discuss the corresponding production modes and 
decay signatures at the LHC, and highlight the requirement for new search strategies. 
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